Computer systems span a variety of architectural designs. Many of these designs have several general features in common. One or more host processors, such as a microprocessor, are connected via a local bus to system memory. The system memory is used to provide the connected processors with program instructions and associated software program data. These programs include at least one operating system and a plurality of application programs.
To transfer data to and from an external storage device, such as a disk or tape drive, an input/output (IO) adapter is normally used. These IO adapters are physically connected to a system bus. The system bus provides a common standardized interface between IO adapters connected to the same bus. The system bus is connected to the local bus so that data can be transferred between the IO adapters and software programs executing upon the host processors. The system memory is used as a repository in the exchange of data. Once contained within system memory, the respective data can be processed by host processors under software program control.
Specific allocation of system memory for IO-related purposes requires that the operating system software makes suitable provisions including file system support, host device driver support, and IO adapter software and hardware support. Application programs must be able to take advantage of this software support.
The nature of this scheme imposes certain limitations upon IO operations. First, the capacity of system memory available specifically for IO operations at any given time is limited. Second, methods to allocate and otherwise use system memory for specifically IO-related purposes must be incorporated in the operating system. Third, the resulting associated performance degradation may limit the number and data rates of storage devices serviced.
One improvement is to incorporate one or more IO adapters into an IO processor (IOP). The IOP contains its own processor capable of executing instructions resulting in the transfer of data between a storage device and the system memory concurrently with application program instructions being executed by a host processor. The IOP may include local memory holding IOP processor instructions, IOP program data, data being transferred, and the like.
Using IOP local memory to hold data being transferred between storage devices and system memory incurs several constraints and disadvantages. First, data transfer operations into and out of the IOP local memory consume a notable amount of bandwidth of the IOP local bus connecting the IOP processor to local memory. Since the IOP processor must also use the IOP local bus for its memory operations, both the data transfer rate and the IOP processor execution rate are affected. Second, use of IOP local memory as a repository for data involved in an IO operation is subject to the same various resource allocation issues and constraints found when system memory is used. The IOP processor must balance the local memory resource between program execution needs and data transfer needs.
Another improvement is "device virtualization". Device virtualization is the facility to present the appearance of a particular type of peripheral storage device to an application program regardless of the actual storage device type. This frees the application program from the need to know any specific details about the storage device. For example, an application program may read from or write to storage as if the storage was composed of one large magnetic disk, when in fact, storage may be comprised of disk drives and tape drives of varying sizes, capabilities, and access logic.
Another example of device virtualization is a RAM disk. The operating system or associated software once again presents the image of a disk drive device. However, the repository for the data associated with the disk device image is maintained within the system memory. This permits faster access to data than would be possible if the data was actually maintained on a disk.
The two examples above illustrate device virtualization implemented "in-board" upon the computer system platform proper. Device virtualization may also be performed "out-board" by the storage device itself. For example, logic within the storage device may make the device appear as one large disk, when it is actually composed of many smaller disks. These physical disk devices may have characteristics such as form factor, capacity, interface protocol, and the like, that are quite dissimilar from the virtual disk device image presented to the computer system.
Both in-board and out-board device virtualization have undesirable properties. In-board device virtualization is performed by the operating system and, hence, requires host processor and system memory resources. Out-board device virtualization generally requires complex and expensive storage devices. Further, data transfer rates may be reduced due to the need to transfer control messages over the relatively slower cabling connecting storage devices with IO adapters.
What is needed is a system that provides device virtualization without requiring specialized storage devices or without utilizing host processor resources. The system should have widespread interoperability, providing a platform independent means of managing specifically IO-related cache memory. The system should be extensible, supporting a wide range of features including data compression, data encryption, global caching, data buffering, and the like.